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DETAILED ACTION 

1 . Claims 1-20 are pending in this application. 

Priority 

2. Receipt is acknowledged of papers submitted under 35 U.S.C. 119(a)-(d), which 
papers have been placed of record in the file. 

Information Disclosure Statement 

3. The information disclosure statement (IDS) submitted on 5/27/05 has been 
considered by the examiner. 

Drawings 

4. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they include the following reference character(s) not mentioned in the 
description: Ref. 460 of Fig. 10, Ref. 565 of Fig. 14 and Ref. 1330 of Fig. 26. These 
appear to have been inadvertently excluded from the specification. For example, it 
seems that Par. 216 refers to the Shot Boundary as Ref. 560, and is merely a typo 
meant to be 565. Corrected drawing sheets in compliance with 37 CFR 1 .121(d), or 
amendment to the specification to add the reference character(s) in the description in 
compliance with 37 CFR 1 .121(b) are required in reply to the Office action to avoid 
abandonment of the application. Any amended replacement drawing sheet should 
include all of the figures appearing on the immediate prior version of the sheet, even if 
only one figure is being amended. Each drawing sheet submitted after the filing date of 
an application must be labeled in the top margin as either "Replacement Sheet" or "New 
Sheet" pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, 
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the applicant will be notified and informed of any required corrective action in the next 
Office action. The objection to the drawings will not be held in abeyance. 

Claim Rejections - 35 USC § 101 

5. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

6. Claims 16, 17 and 19 rejected under 35 U.S.C. 101 because the claimed 
inventions are directed to non-statutory subject matter. 

Claim 16 fails to fall within a statutory category of invention. It is directed to the 
program code itself, not a process occurring as a result of executing the program code, 
a machine programmed to operate in accordance with the program code, a composition 
of matter, nor a manufacture structurally and functionally interconnected with the 
program code in a manner which enables the program to act as a computer component 
and realize its functionality. Therefore, it is non-statutory under 35 USC 101 . 

Claim 17 fails to be limited to embodiments that fall within a statutory category of 
invention. In Claim 17 the "providing medium" can be interpreted as "a transmission 
medium" as evidenced by Claim 19 and therefore is not statutory. 

Claim 19 fails to fall within a statutory category of invention. A "transmission 
medium" does not constitute a manufacture nor does it fall within any of the other three 
statutory categories. 

While "functional descriptive material" may be claimed as a statutory product 
(i.e., a "manufacture") when embodied on a tangible computer readable medium, a 
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signal or transmission media embodying that same functional descriptive material is 
neither a process nor a product (i.e., a tangible "thing") and therefore does not fall within 
one of the four statutory classes of § 1 01 . Rather, a signal or transmission media is a 
form of energy, in the absence of any physical structure or tangible material. Thus, the 
claim is not limited to statutory embodiments. See MPEP 2106. 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

8. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 
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9. Claims 1 -1 2 and 1 4-20 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Snook, US Patent 6400378, and further in view of Dufaux, US Patent 
6711587. 

As in Claim 1, Snook teaches a media handling system in which candidate video 
sequences are displayed on a display screen in schematic form for selection by a user 
(Fig. 3 and corresponding text), the system comprising: a display screen for displaying 
representations of the candidate video sequences for selection by a user (ref. 120), 
each representation including one or more images derived from the respective video 
sequences (Col. 2, lines 64-66); and a user control for defining a set of one or more of 
the video sequences (Col. 2, line 55-Col. 3, line 1). While Snook teaches displaying 
video sequences, represented by a keyframe thumbnail, that may be for selected by a 
user to define a set of one or more video sequences, they fail to show the means for 
detecting human faces in the candidate video sequences to represent the candidate 
video sequences as a keyframe as recited in the claims. In the same field of the 
invention, Dufaux teaches a media handling system similar to that of Snook. In 
addition, Dufaux further teaches means for detecting human faces in the candidate 
video sequences (Fig. 5, ref. 500 and corresponding text); and assigning them as the 
key frame representations of the candidate video sequences (Col. 4, lines 4-7). It would 
have been obvious to one of ordinary skill in the art, having the teachings of Snook and 
Dufaux before him at the time the invention was made, to modify displaying video 
sequences, represented by a keyframe thumbnail, that may be for selected by a user to 
define a set of one or more video sequences taught by Snook to include the means for 
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detecting human faces in the candidate video sequences and assigning them as the key 
frame representations of the candidate video sequences of Dufaux, in order to obtain 
means for detecting human faces in candidate video sequences, displaying video 
sequences, represented by an image representing human faces derived from the 
respective video sequences, that may be for selected by a user to define a set of one or 
more video sequences. One would have been motivated to make such a combination 
because a way for the system to select a more relevant, user friendly representative 
image would have been obtained, as taught by Dufaux. 

As in Claim 2, Snook teaches the set of one or more of the video sequences is 
an ordered edited set forming an output media product (Col. 2, lines 62-64). 

As in Claim 3, the above combination of Snook and Dufaux teach the system of 
Claim 1 as rejected supra. Snook further teaches a further ordered representation of a 
group of at least a subset of the video sequences forming the output media product 
(Snook, Col. 8, lines 2-3). Snook fails to explicitly teach images representing human 
faces derived from the respective video sequences in the group. Dufaux teaches 
detected human faces represent their respective video sequences (Dufaux, Col. 4, lines 
4-7). It would have been obvious to one of ordinary skill in the art, having the teachings 
of Snook and Dufaux before him at the time the invention was made, to modify 
displaying video sequences, represented by a keyframe thumbnail, that may be for 
selected by a user to define a set of one or more video sequences and a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product taught by Snook to include the means for detecting human faces in the 
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candidate video sequences and assigning them as the key frame representations of the 
candidate video sequences of Dufaux, in order to obtain a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product, the video sequences represented by an image representing human 
faces derived from the respective video sequences. One would have been motivated to 
make such a combination because a way for the system to select a more relevant, user 
friendly representative image would have been obtained, as taught by Dufaux. 

As in Claim 4, Snook teaches the ordered representation is a timeline 
representation ("video timeline", Col. 6, line 29), providing an ordered representation of 
the group of video sequences forming the output media product along a generally 
rectilinear path on the display screen (Fig. 6A and corresponding text). 

As in Claim 5, Snook teaches which the ordered representation may be scaled 
so as to vary the proportion of the video sequences forming the output media product 
which are currently displayed in the ordered representation (Fig. 6A ref. 625 and 
corresponding text). 

As in Claim 6, Snook fails to explicitly teach detecting human faces derived from 
the respective video sequences and displaying the one with the highest probability of 
containing a face in the group. Dufaux teaches the detecting means is operable to 
detect a probability of a human face being present in each field or frame of the video 
sequences (Col. 1, lines 55-57); each displayed representation of a candidate video 
sequence including one or more images representing human faces which have the 
highest probability levels amongst the respective video sequences (Col. 12, line 4 et 
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seq.). It would have been obvious to one of ordinary skill in the art, having the teachings 
of Snook and Dufaux before him at the time the invention was made, to modify 
displaying video sequences, represented by a keyframe thumbnail, that may be for 
selected by a user to define a set of one or more video sequences and a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product taught by Snook to include the detecting human faces derived from the 
respective video sequences and displaying the one with the highest probability of 
containing a face in the group of Dufaux, in order to obtain a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product, the video sequences represented by an image with the highest 
probability of containing human faces derived from the respective video sequences. 
One would have been motivated to make such a combination because a way for the 
system to select a more relevant, user friendly representative image would have been 
obtained, as taught by Dufaux. 

As in Claim 7, Snook fails to explicitly teach weighting frames according to the 
size of detected human faces derived from the respective video sequences and 
displaying the one with the highest probability of containing a face in the group. Dufaux 
teaches the detecting means is operable to detect a probability of a human face being 
present in each field or frame of the video sequences (Fig. 5) and to weight at least 
some of the detected probability levels in dependence on the size of the detected face 
(Col. 12, lines 4 et seq.); each displayed representation of a candidate video sequence 
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including one or more images representing human faces which have the highest 
weighted probability levels amongst the respective video sequences (Col. 12, line 4 et 
seq.). It would have been obvious to one of ordinary skill in the art, having the teachings 
of Snook and Dufaux before him at the time the invention was made, to modify 
displaying video sequences, represented by a keyframe thumbnail, that may be for 
selected by a user to define a set of one or more video sequences and a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product taught by Snook to include the weighting frames according to the size of 
detected human faces derived from the respective video sequences and displaying the 
one with the highest probability of containing a face in the group of Dufaux, in order to 
obtain a further ordered representation of a group of at least a subset of the video 
sequences forming the output media product, the video sequences represented by an 
image with the highest probability of containing a particular sized human faces derived 
from the respective video sequences. One would have been motivated to make such a 
combination because a way for the system to select a more accurate face detection 
method for keyframe selection would have been obtained, as taught by Dufaux. 

As in Claim 8, Dufaux teaches the detecting means is operable to weight the 
probability levels so that detected faces closer in size to a desired representation size 
are more likely to be selected to form a displayed representation (Col. 2, lines 12-25). 

Snook fails to explicitly teach weight the probability levels so that detected faces 
closer in size to a desired representation size are more likely to be selected to form a 
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displayed representation. Dufaux teaches the detecting means is operable to detect a 
probability of a human face being present in each field or frame of the video sequences 
(Fig. 5) and to weight at least some of the detected probability levels in dependence on 
the size of the detected face (Col. 12, lines 4 et seq.); each displayed representation of 
a candidate video sequence including one or more images representing human faces 
which have the highest weighted probability levels amongst the respective video 
sequences (Col. 12, line 4 et seq.). It would have been obvious to one of ordinary skill in 
the art, having the teachings of Snook and Dufaux before him at the time the invention 
was made, to modify displaying video sequences, represented by a keyframe 
thumbnail, that may be for selected by a user to define a set of one or more video 
sequences and a further ordered representation of a group of at least a subset of the 
video sequences forming the output media product taught by Snook to include the 
weight the probability levels so that detected faces closer in size to a desired 
representation size are more likely to be selected to form a displayed representation 
Dufaux, in order to obtain a further ordered representation of a group of at least a 
subset of the video sequences forming the output media product, the video sequences 
represented by an keyframe image determined according to weight, weight the 
probability levels so that detected faces closer in size to a desired representation size 
are more likely to be selected to form a displayed representation. One would have been 
motivated to make such a combination because a way for the system to select a more 
accurate facial detection method for keyframe selection would have been obtained, as 
taught by Dufaux. 
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As in Claim 9, Snook fails to explicitly teach the detecting means is operable to 
apply the weighting over a subset of the fields or frames of a video sequence. Dufaux 
teaches the detecting means is operable to apply the weighting over a subset of the 
fields or frames of a video sequence (Col. 2, lines 20-24). It would have been obvious to 
one of ordinary skill in the art, having the teachings of Snook and Dufaux before him at 
the time the invention was made, to modify displaying video sequences, represented by 
a keyframe thumbnail, that may be for selected by a user to define a set of one or more 
video sequences and a further ordered representation of a group of at least a subset of 
the video sequences forming the output media product taught by Snook to include the 
weighting frames according to the size of detected human faces derived from the 
respective video sequences and displaying the one with the highest probability of 
containing a face in the group of Dufaux, in order to obtain a further ordered 
representation of a group of at least a subset of the video sequences forming the output 
media product, the video sequences represented by a detected keyframe image, the 
detecting means is operable to apply the weighting over a subset of the fields or frames 
of a video sequence. One would have been motivated to make such a combination 
because a way for the system to select a more accurate face detection method for 
keyframe selection would have been obtained, as taught by Dufaux. 

As in Claims 10-11, Snook fails to explicitly teach the weighting over frames of a 
video sequence. Dufaux teaches the detecting means is operable to apply the 
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weighting over frames of a video sequence (Col. 2, lines 20-24). It would have been 
obvious to one of ordinary skill in the art, having the teachings of Snook and Dufaux 
before him at the time the invention was made, to modify displaying video sequences, 
represented by a keyframe thumbnail, that may be for selected by a user to define a set 
of one or more video sequences and a further ordered representation of a group of at 
least a subset of the video sequences forming the output media product taught by 
Snook to include the detecting means is operable to apply the weighting over frames of 
a video sequence of Dufaux, in order to obtain a further ordered representation of a 
group of at least a subset of the video sequences forming the output media product, the 
video sequences represented by a detected keyframe image, the detecting means is 
applys the weighting over frames of a video sequence. One would have been 
motivated to make such a combination because a way for the system to select a more 
accurate face detection method for keyframe selection would have been obtained, as 
taught by Dufaux. 

As in Claim 12, Snook teaches teaches selection of a displayed representation 
by the user control causes the display of the corresponding video sequence (Col. 3, 
lines 33-37). 

As in Claim 14, Snook fails to explicitly teach indicating that faces detected in two 
or more respective ones of the candidate video sequences represent the same person's 
face. Dufaux teaches a user control for indicating that faces detected in two or more 
respective ones of the candidate video sequences represent the same person's face 
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(Fig. 5 and corresponding text). It would have been obvious to one of ordinary skill in 
the art, having the teachings of Snook and Dufaux before him at the time the invention 
was made, to modify displaying video sequences, represented by a keyframe 
thumbnail, that may be for selected by a user to define a set of one or more video 
sequences and a further ordered representation of a group of at least a subset of the 
video sequences forming the output media product taught by Snook to include the 
indicating that faces detected in two or more respective ones of the candidate video 
sequences represent the same person's face of Dufaux, in order to obtain a further 
ordered representation of a group of at least a subset of the video sequences forming 
the output media product, the video sequences represented by a detected keyframe 
image with detected faces and indicating that faces detected in two or more respective 
ones of the candidate video sequences represent the same person's face. One would 
have been motivated to make such a combination because a way for the system to 
select a more accurate face detection method for keyframe selection would have been 
obtained, as taught by Dufaux. 

As in Claim 15, Snook teaches a method of media handling system in which 
candidate video sequences are displayed on a display screen in schematic form for 
selection by a user (Fig. 3 and corresponding text), the method comprising the steps of: 
displaying on a display screen representations of the candidate video sequences for 
selection by a user (ref. 120), each representation including one or more images 
derived from the respective video sequences (Col. 2, lines 64-66); and providing a user 
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control for defining a set of one or more of the video sequences (Col. 2, line 55-Col. 3, 
line 1). While Snook teaches displaying video sequences, represented by a keyframe 
thumbnail, that may be for selected by a user to define a set of one or more video 
sequences, they fail to show the means for detecting human faces in the candidate 
video sequences to represent the candidate video sequences as a keyframe as recited 
in the claims. In the same field of the invention, Dufaux teaches a media handling 
method similar to that of Snook. In addition, Dufaux further teaches means for detecting 
human faces in the candidate video sequences (Fig. 5, ref. 500 and corresponding text); 
and assigning them as the key frame representations of the candidate video sequences 
(Col. 4, lines 4-7). It would have been obvious to one of ordinary skill in the art, having 
the teachings of Snook and Dufaux before him at the time the invention was made, to 
modify displaying video sequences, represented by a keyframe thumbnail, that may be 
for selected by a user to define a set of one or more video sequences taught by Snook 
to include the means for detecting human faces in the candidate video sequences and 
assigning them as the key frame representations of the candidate video sequences of 
Dufaux, in order to obtain means for detecting human faces in candidate video 
sequences, displaying video sequences, represented by an image representing human 
faces derived from the respective video sequences, that may be for selected by a user 
to define a set of one or more video sequences. One would have been motivated to 
make such a combination because a way for the system to select a more relevant, user 
friendly representative image would have been obtained, as taught by Dufaux. 
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As in Claim 16, Snook teaches computer software having program code (Snook, 
Col. 2, lines 58-60) for carrying out the method of Claim 15 (See Claim 15 rejection 
supra). 

As in Claim 17, Snook teaches a providing medium (Snook, Fig. 2 and 
corresponding text) for providing program code according to claim 16 (See Claim 16 
rejection supra). 

As in Claim 18, Snook teaches the medium being a storage medium (Snook, Fig. 
2 and corresponding text). 

As in Claim 19, the above combination of Snook and Dufaux teach the method of 
Claim 15 as rejected supra. Snook fails to explicitly teach a transmission medium for 
providing program code. Dufaux teaches the computer readable medium capable of 
being a transmission medium (Col. 13, lines 65-67). It would have been obvious to one 
of ordinary skill in the art, having the teachings of Snook and Dufaux before him at the 
time the invention was made, to modify the providing medium of Snook to include the 
transmission medium of Dufaux in order to obtain a transmission of code from a remote 
source to perform the aforementioned method. One would have been motivated to 
combine Dufaux and Snook for reasons explained in the rejection of Claim 15, supra. 

As in Claim 20, Snook teaches a media handling system in which candidate 
video sequences are displayed on a display screen in schematic form for selection by a 
user (Fig. 3 and corresponding text), the system comprising: a display screen for 
displaying representations of the candidate video sequences for selection by a user (ref. 
120), each representation including one or more images derived from the respective 
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video sequences (Col. 2, lines 64-66); and a user control for defining a set of one or 
more of the video sequences (Col. 2, line 55-Col. 3, line 1). While Snook teaches 
displaying video sequences, represented by a keyframe thumbnail, that may be for 
selected by a user to define a set of one or more video sequences, they fail to show the 
detector to detect human faces in the candidate video sequences to represent the 
candidate video sequences as a keyframe as recited in the claims. In the same field of 
the invention, Dufaux teaches a media handling system similar to that of Snook. In 
addition, Dufaux further teaches detector to detect human faces in the candidate video 
sequences (Fig. 5, ref. 500 and corresponding text); and assigning them as the key 
frame representations of the candidate video sequences (Col. 4, lines 4-7). It would 
have been obvious to one of ordinary skill in the art, having the teachings of Snook and 
Dufaux before him at the time the invention was made, to modify displaying video 
sequences, represented by a keyframe thumbnail, that may be for selected by a user to 
define a set of one or more video sequences taught by Snook to include the detector to 
detect human faces in the candidate video sequences and assigning them as the key 
frame representations of the candidate video sequences of Dufaux, in order to obtain a 
detector to detect human faces in candidate video sequences, displaying video 
sequences, represented by an image representing human faces derived from the 
respective video sequences, that may be for selected by a user to define a set of one or 
more video sequences. One would have been motivated to make such a combination 
because a way for the system to select a more relevant, user friendly representative 
image would have been obtained, as taught by Dufaux. 
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10. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over Snook, 
US Patent 6400378, Dufaux, US Patent 6711587 and further in view of Trivedi et al., US 
Patent Application Publication 2006/0187305, hereinafter Trivedi. 

Snook and Dufaux teach means for detecting human faces in candidate video 
sequences, displaying video sequences, represented by an image representing human 
faces derived from the respective video sequences, that may be for selected by a user 
to define a set of one or more video sequences (Claim 1 rejection supra). While Snook 
and Dufaux teach displaying video sequences, represented by a keyframe thumbnail, 
that may be for selected by a user to define a set of one or more video sequences, 
detecting human faces in the candidate video sequences to represent the candidate 
video sequences as a keyframe, they fail to show the candidate video sequences are 
selected from a video sequence captured by a surveillance camera as recited in the 
claims. In the same field of the invention, Trivedi teaches a media handling system and 
facial detection similar to that of Snook and Dufaux. In addition, Trivedi further teaches 
in which the candidate video sequences are selected from a video sequence captured 
by a surveillance camera (Par. 0038). It would have been obvious to one of ordinary 
skill in the art, having the teachings of Snook, Dufaux before him at the time the 
invention was made, to modify displaying video sequences, represented by a keyframe 
thumbnail, that may be for selected by a user to define a set of one or more video 
sequences, detecting human faces in the candidate video sequences and assigning 
them as the key frame representations of the candidate video sequences taught by 
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Snook and Dufaux to include the candidate video sequences are selected from a video 
sequence captured by a surveillance camera of Trivedi, in order to obtain means for 
detecting human faces in candidate video sequences captured by a surveillance 
camera, displaying video sequences, represented by an image representing human 
faces derived from the respective video sequences, that may be for selected by a user 
to define a set of one or more video sequences. One would have been motivated to 
make such a combination because a way for police and security guards to single out 
people from video surveillance would have been obtained, as taught by Trivedi. 

The prior art made of record on form PTO-892 and not relied upon is considered 
pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1 .1 1 1 (c) to 
consider these references fully when responding to this action. The documents cited 
therein teach similar video editing and keyframe selection techniques as well as 
methods for detecting human faces within video. 



Application/Control Number: 10/536,814 Page 19 

Art Unit: 2179 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Sara M. Hanne whose telephone number is (571) 272- 
4135. The examiner can normally be reached on M-F 7:30am-4:00pm, off on 
alternating Fridays. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, WEILUN LO can be reached on (571) 272-4847. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

smh 



/Weilun Lo/ 

Supervisory Patent Examiner, Art Unit 2179 



